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In  Naval  Aviation  flight  test  of  the  1940’s  and  50’s,  test  pilots  were  selected 
almost  exclusively  from  the  fighter  and  attack  communities.  Then,  as  now,  the  test  pilot 
was  a  fleet  aviator  first  and  a  test  pilot  second.  Tour  lengths  were  fairly  short,  2  to 
3  years,  then  the  pilots  went  back  to  the  fleet  to  keep  up  with  the  changes  taking  place  on 
the  “front  lines.”  As  in  any  organization,  flight  test  took  on  the  flavor  of  its  pilots,  in  this 
case,  single  seat  and  dependent  upon  themselves  for  survival.  These  fighter  pilots  tested 
all  aircraft  destined  for  use  in  Naval  Aviation  from  the  F-4U  Corsair  to  the  P-2V 
Neptune. 


The  Naval  Air  Test  Center  was  divided  into  several  Test  Directorates,  including 
Flight  Test  and  Systems  Test.  The  test  pilots  were  trained  to  perform  either  flight  test  or 
systems  test,  rarely  both.  The  plane  was  thought  of  separately  from  the  systems  it  used  to 
do  its  mission.  The  Flight  Test  Directorate  tested  whatever  aircraft  industry  produced. 
Test  pilots  traveled  to  the  contractor’s  facility  to  leam  about  the  new  planes  they  were 
about  to  get,  rarely  to  influence  the  designs  or  system  integration.  The  test  pilot  and 
flight  test  engineers  received  their  ships  from  industry,  and  worked  for  months,  even 
years,  identifying  deficiencies  that  they  found  on  “finished  products”  during  what  was 
called  the  “developmental  phase”  of  flight  test.  These  deficiencies  were  fed  back  to  the 
designers,  changes  were  made  to  accommodate  or  fix  these  problems,  and  the  plane  was 
sent  back  for  more  flight  test.  We  called  this  “kicking  it  over  the  fence.”  The  designers 
and  builders  would  throw  it  over  the  fence  to  the  testers,  who  would  test  it  and  throw  it 
back  to  fix  the  problems  that  they  found.  The  process  continued  until  the  aircraft  was 
“acceptable”  for  the  fleet,  as  determined  by  the  test  pilots  that  flew  them,  or  it  was 
cancelled. 


Planes  were  much  less  sophisticated  and  cheaper  (even  given  inflation)  to  build  in 
those  days,  and  we  changed  Type/Model/Series  (TMS)  in  fleet  squadrons  several  times 
per  decade.  Accident  rates  were  much  higher  then  too,  averaging  50-60  aircraft  lost  per 
100,000  flight-hours.  Mishaps  were  viewed  as  a  part  of  flying.  As  technology  advanced 
and  planes  became  more  and  more  sophisticated  in  design,  changes  discovered  in 
developmental  flight  test  phases  became  more  and  more  expensive  to  correct.  Some 
problems  were  not  fixed  because  they  were  too  expensive  to  implement  in  the  production 
line  and  retrofit  throughout  the  fleet.  These  deficiencies  were  typically  corrected  in  the 
next  production  version  of  the  plane,  or  the  next  plane  designed  and  built  by  that 
company.  Deficiencies  not  fixed  often  generated  changes  in  procedures  that  “worked 
around”  the  deficiency. 


There  are  many  planes  flying  in  the  fleet  today  that  have  basic  design  or 
functionality  deficiencies,  limiting  the  operational  capabilities  or  creating  human  factors 
problems  that  relate  directly  to  flight  safety.  While  the  test  pilots  are  trained  to 
qualitatively  determine  the  potential  impact  of  deficiencies,  it  is  extremely  difficult  to 
quantify  the  potential  down  line  risk  to  plane  and  pilot,  or  estimate  the  total  cost  of 
leaving  deficiencies  in  the  planes.  How  do  you  quantify  losses  because  of  having  the 
pilots  “workaround”  these  problems  while  attempting  other  high  gain  tasks  such  as  air-to- 
air  combat,  weapons  deliveries,  or  executing  emergency  procedures?  How  do  you 


quantify  just  how  many  planes  will  be  (or  were)  lost  due  to  “pilot  error”  because 
deficiencies  were  left  in  planes  and  operational  workarounds  were  developed  instead? 
While  these  were  difficult  to  impossible  tasks  to  quantify,  the  ever  rising  cost  of 
correcting  deficiencies  was  not. 

As  time  progressed,  engineers  began  to  realize  that  one  of  the  ways  to  reduce  the 
cost  to  fix  these  deficiencies  was  to  try  and  catch  them  earlier,  perhaps  as  early  as  the 
design  phase.  This  led  to  the  creation  of  a  Preliminary  Design  Review  (PDR)  and  a 
Critical  Design  Review  (CDR)  process  where  teams  could  influence  the  design  process 
very  early  on,  many  times  long  before  metal  was  bent  and  the  plane  or  system  was  still  on 
paper.  (This  concept  was  very  successful  from  a  manager’s  view  point,  and  on  some 
programs  was  expanded  to  include  Quarterly  Design  Reviews  and  Technical  Interface 
Meetings  (TIM’s).)  For  years,  it  was  rare  for  a  test  pilot  and/or  test  engineer  to  be 
included  in  these  meetings,  which  were  usually  covered  by  program  office  personnel. 
Officers  from  each  community  with  an  engineering  background,  known  as  “Class 
Desks,”  were  responsible  to  the  program  office  for  the  engineering  influence  on  all 
developmental  aircraft  and  systems.  Even  if  test  pilots  and  engineers  were  invited  to 
these  events,  rarely  could  they  go  because  of  the  workload  of  existing  products  lined  up 
for  test.  Sometimes  nontest,  pilot-trained  fleet  aviators  were  used  to  get  direct  inputs  on 
new  systems  and  design  issues.  This  lack  of  participation  led  to  us  not  being  “invited”  to 
attend  and  helped  create  an  isolationist  mentality  between  flight  test  and  the  program 
office. 


This  isolationist  attitude  became  almost  the  norm  in  test  pilots  and  engineers,  and 
we  became  “those  test  guys.”  While  we  did  great  work  for  the  fleet  and  the  program 
office,  we  were  often  blamed  for  delays  caused  by  system  short  comings,  because  they 
were  discovered  during  test.  The  test  community  began  to  get  a  reputation  in  the 
program  offices  for  being  a  stumbling  block  or  hurdle  to  be  jumped  rather  than  a  valuable 
member  of  the  team,  striving  to  get  quality  products  into  the  hands  of  the  fleet  user. 
Testing  that  the  program  office  saw  no  need  for  became  known  as  “science  projects.”  As 
this  rift  grew,  the  test  community  became  more  standoff-ish.  The  Test  Pilot  School 
(TPS)  taught  us  to  be  “above  the  team.”  We  began  to  use  words  like  “within  the  scope  of 
these  tests”  in  reports.  To  the  program  office,  this  translated  to  “any  other  problems  that 
you  may  find  while  using  this  product  were  not  discovered  because  those  cheap  skates  at 
the  program  office  didn’t  fund  us  to  test  everything!”  It  was  also  drilled  into  us  that  we 
could  never  recommend  a  fix  to  the  deficiency,  no  matter  how  obvious,  because  the 
program  office  (and  the  contractor)  might  hold  us  liable  if  the  fix  didn’t  work  or  was 
incorrectly  implemented.  The  tighter  money  became,  the  more  the  program  office  put 
pressure  on  the  test  community  to  “stop  testing  and  just  get  it  out  the  door.” 

As  we  moved  through  the  1960’s  and  70’s  into  the  80’s,  aircraft  cost  per  unit 
soared  and  new  model  production  slowed  dramatically.  Aircraft  TMS  stayed  in 
squadrons  longer.  Test  pilots  began  coming  from  their  own  fleet  communities.  They 
brought  with  them  the  specific  mission  interests  of  their  community.  They  tested  the 
aircraft  and  systems  that  were  needed  to  carry  out  their  missions.  The  Test  Directorates 
reorganized  into  “like  platform”  organizations.  Strike  Aircraft  Test  Directorate  picked  up 
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all  the  fighter  and  attack  platforms.  Force  Aircraft  Test  Directorate  (then  ASW  Test 
Directorate)  picked  up  the  large  aircraft,  and  Rotary  Wing  Aircraft  Test  Directorate  all 
the  helicopter  types.  TPS  went  to  a  fixed-wing  pilot  curriculum,  a  rotary-wing  pilot 
curriculum,  and  a  systems  curriculum.  All  pilots  were  trained  as  fixed-  or  rotary-wing 
test  pilots,  with  a  healthy  dose  of  systems  test  knowledge  thrown  in.  The  engineers  and 
Naval  Flight  Officers  (NFO’s)  were  trained  in  the  systems  curriculum.  As  before,  the  test 
directorates  took  on  the  flavor  of  the  communities  that  the  test  pilots  came  from. 

In  the  fleet,  changes  were  occurring  too.... 

As  the  fighter  and  attack  communities  developed  and  institutionalized  their 
single-seat  mentality,  large  fleet  aircraft  communities  (such  as  the  Airborne  Early 
Warning  and  Patrol)  and  helicopter  communities  were  developing  crew  concepts.  Dollar 
cost  of  mishaps  was  adding  up  fast,  and  flight  safety  began  driving  significant  changes  in 
Naval  Aviation  communities.  Angled  flight  decks  were  in,  straight  decks  were  out.  We 
stood  up  the  Naval  Safety  Center,  and  the  Naval  Aviation  Training  and  Operational 
Procedures  Standardization  programs  began  in  the  60’ s.  The  new  Safety  Center  analyzed 
mishaps  and  made  recommendations  to  the  communities.  Accident  rates  began  to  drop 
dramatically  across  the  fleet. 


Naval  Aviation  Mishap  Rate 

FY  50  -  98 


60  - 

50  - 

g 

o  2 

i-H  a 

40  - 

u  © 

30  - 

vi 

cl 

«  .££ 
E 

20  - 

10  - 

776  aircraft 
destroyed  in 
1954  ^ 


„  Angled  decks 
-Aviation  Safety  Center 


33  aircraft 
destroyed  in 
1998  , 


'Naval  Aviation  Maintenance  Program 
(NAME),  1956 

V.G  concept  Initiated 
'NATO  PS  Program,  1961 

Squadron  Safety  Program 
if  System  Safety  Designated  Aircraft 
^  ''x  Act 


- 1 - 1 - 1 - 1 - 

50  54  58  62  66 


70  74  78  82  86  90  94  98 


Fiscal  Year 


3 


In  large  aircraft  mishaps,  where  the  crews  survived,  interviews  showed  that  many 
were  caused  by  the  breakdown  of  crew  communications  during  emergency  situations. 
The  guy  with  the  right  answer  was  not  in  the  information/decision  loop,  and  uninformed 
or  bad  decisions  were  made.  In  an  effort  to  improve  this,  a  Crew  Resource  Management 
(CRM)  program  was  started  in  the  early  1980’s.  CRM  trained  crews  to  work  better 
together,  through  the  proper  use  of  teaming  techniques.  These  techniques  by  no  means 
relieve  the  aircraft  commanders  or  mission  commanders  of  their  responsibilities  as  the 
officer  in  charge,  but  allow  an  avenue  for  the  “other  guys”  to  voice  their  concerns  and 
present  possible  solutions.  This  shift  was  slow  to  be  accepted  in  the  fighter/attack 
communities,  because  they  “don’t  have  a  crew.”  However,  the  decline  in  mishaps  was 
hard  to  refute  especially  in  the  helo  communities.  Over  several  years,  CRM  became 
known  as  Aircrew  Coordination  Training  (ACT)  and  it  is  now  spreading  to  all  of  Naval 
Aviation  and  into  the  USAF.  Each  TMS  in  the  Navy  now  has  its  own  aircraft  specific 
ACT  program.  As  a  direct  result  of  . this  and  other  programs,  1999  is  proving  to  be  the 
safest  year  ever  on  record  in  Naval  Aviation. 

....Meanwhile,  back  in  flight  test.... 

Because  the  test  directorates  were  now  a  microcosm  of  their  fleet  communities, 
no  where  was  this  crew  philosophy  change  more  obvious  than  the  large  aircraft  test 
community  where  crews  and  CRM  were  already  the  norm.  Their  planes  were  all 
multicrewed,  and  the  pilots  were  accustomed  to  everyone  doing  their  job  and  being  “an 
equal”  member  of  the  crew/team  from  a  mission  perspective.  The  test  flightcrew 
naturally  became  a  team  as  well,  including  not  just  the  test  pilot  and  flight  test  engineer, 
but  the  “back  end”  operators,  the  technicians,  and  the  maintenance  crews  that  supported 
them.  The  test  pilot  and/or  test  NFO  assigned  began  to  act  as  a  “project  officer,”  not  only 
working  programmatic  issues  within  the  team  but  developing,  managing,  and  flying  the 
flight  tests  for  the  project  as  well.  All  members  of  the  team  had  their  say  on  issues 
ranging  from  the  conduct  of  the  test,  to  the  data  collected,  and  the  test  techniques  used  to 
collect  it.  More  and  more,  the  test  pilots  worked  as  a  team  with  the  program  office, 
attending  PDR  and  CDR  and  various  Technical  Interchange  Meetings  (TIM’s.)  More 
and  more,  the  teams  worked  directly  with  industry  to  develop  aircraft  and  systems. 
Gradually,  “up  front  and  early”  became  the  norm,  rather  than  “kick  it  over  the  fence.” 

In  the  early  1990’s,  as  a  result  of  the  Base  Realignment  and  Closure  Commissions 
(BRACC),  we  combined  the  Science  and  Technology  pieces  of  the  development  process 
with  Test  and  Evaluation  and  moved  the  Naval  Air  Development  Center  from 
Warminster,  Pennsylvania,  to  the  Naval  Air  Test  Center  in  Patuxent  River,  Maryland.  At 
the  same  time,  another  realignment  began  at  the  Naval  Air  Systems  Command  and  the 
Test  Center  (now  the  Naval  Air  Warfare  Center  or  NAWC.)  NAVAIR  became  known  as 
the  Naval  Air  Systems  Team  (NAST)  and  realigned  itself  into  Program  Teams.  The  first 
platform  to  fully  use  this  was  the  E-2C. 

The  move  from  Warminster  allowed  the  flying  laboratories  that  developed  new 
systems,  and  researched  new  technologies,  to  be  directly  integrated  with  the 
developmental  test  community.  Often,  our  test  pilots  flew  these  flying  laboratories  and 
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could  interface  directly  with  the  scientists  that  were  working  a  particular  project.  The 
NAVAIR/NAWC  reorganization  into  Program  Teams  allowed  for  the  development  of 
Integrated  Product  Teams,  each  drawing  members  from  throughout  the  NAVAIR 
competency  matrix  into  their  individual  teams,  under  the  cognizance  of  a  Program 
Manager.  Soon,  members  of  the  team  came  from  all  parts  of  the  aviation  community, 
industry,  the  program  office,  and  the  Research  Development,  Test,  and  Evaluation 
community  as  well.  Test  pilots  and  engineers  flying  these  research  projects  took 
information  and  data  back  to  the  program  office,  influencing  the  current  and  future 
programs  that  the  research  would  soon  touch.  They  also  fed  valuable  data  into  the  R&D 
community  about  fleet  needs  and  desires  as  these  experimental  systems  continued  to 
develop. 

Teaming  now  occurs  on  many  levels  within  the  overall  NAVAIRSYSCOM 
Team.  The  program  office’s  each  have  their  own  Program  Operating  Guides,  which 
show  the  lay  down  of  each  of  their  product  teams.  Within  the  test  community,  teams  are 
created  and  supported  based  on  the  cost,  schedule,  and  performance  requirements 
established  by  the  customer  (usually  the  program  office).  Rapid  prototyping  teams  are 
usually  very  high  priority,  work  over  a  very  short-time  period,  and  encompass  a  large 
number  of  people.  In  the  late  1980’s  during  the  Persian  Gulf  conflicts,  the  P-3 
community  was  doing  littoral  surveillance  and  sea  control  with  their  new  ISAR  radar.  At 
that  time,  the  P-3,  normally  an  open  ocean  ‘blue  water  ops’  aircraft,  had  few  survivability 
features.  Yet,  she  was  operating  in  the  tightly  confined  waters  of  the  Persian  Gulf  flying 
over  threats  on  night  reconnaissance  missions.  With  one  squadron  already  on  patrol,  the 
next  was  in  work-ups  up  to  relieve  them  in  theatre.  A  rapid  prototype  team  was 
established  to  modify  the  planes  of  the  next  squadron  to  go.  They  had  the  daunting  task 
of  developing,  installing,  and  testing  four  survivability  enhancements  in  test  aircraft,  then 
installing  these  features  in  five  fleet  planes  within  6  weeks.  The  team  that  was 
established  was  run  entirely  by  the  test  community.  Test  pilots  and  test  engineers, 
experts  on  the  P-3  platform  and  the  survivability  systems  being  installed,  teamed  with 
system’s  manufactures  and  industry  and  were  completely  supported  by  the  program 
office.  Facilities  and  capabilities  throughout  NAST  were  used  to  install,  test,  and  retrofit 
the  modifications.  Surprisingly,  communications  with  the  receiving  squadron  and 
training  for  her  crews  proved  the  toughest  of  the  tasks,  as  the  team  raced  to  solve  the 
monumental  engineering,  design,  and  installation  challenges.  The  final  plane  was 
completed  1  day  ahead  of  schedule  and  left  Patuxent  River  for  the  deployed  site. 

On  a  large  program  team,  like  the  P-3  Antisurface  Warfare  Improvement 
Program,  smaller  elite  teams  form  the  nucleus  for  individual  tests,  while  a  single  project 
officer  manages  the  overall  testing  efforts.  A  flight  test  team  would  form  up  for  a 
weapons  test  program,  for  example,  that  included  members  from  maintenance  and  range 
support  and  services.  Flightcrews  would  include  the  project  officer  (usually  a  test  pilot  or 
test  NFO)  and  qualified  aircrewmen.  Test  engineers  familiar  with  the  platform  would  be 
included,  as  well  as  others  who  are  experts  on  the  weapon  itself.  Depending  on  the  stage 
of  development,  the  team  would  travel  to  the  contractor’s  facility  to  interface  with  the 
developers,  as  in  a  new  start,  or  for  the  integration  of  an  existing  weapon,  the  developers 
would  meet  the  team  at  the  integration  facility.  All  members  of  the  test  team  have  a  “no 
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vote.”  The  no  vote  is  the  individual’s  ability  to  stop  a  flight  test  program  for  a  limited 
duration  if  some  piece  under  their  cognizance  didn’t  measure  up  or  was  creating 
confusion.  After  a  “reasonable  man”  review  of  the  safety  or  systems  issues,  the  project 
continues.  Project  officers  stay  with  the  program  for  2  to  3  years  and  work  primarily  that 
program,  providing  stability  and  continuity  to  the  team. 

Where  is  teaming  going  in  the  future?  Again,  in  the  large  aircraft  arena,  we  are 
looking  at  the  inclusion  of  the  Naval  Research  Labs  (NRL)  flight  research  detachment 
into  the  Force  Aircraft  Test  Squadron.  Test  pilots  and  engineers  will  begin  flying  with 
other  researchers  that  were,  in  the  past,  isolated  within  the  confines  of  NRL.  Teams  of 
test  pilots  and  engineers  are  working  directly  with  industry  on  the  Joint  Strike  Fighter 
(JSF)  program.  JSF  is  the  next  generation  aircraft  intended  to  support  the  role  of  fighter 
and  attack  aircraft  for  the  Navy  and  Air  Force.  Throughout  the  test  communities,  test 
pilot  and  project  officer  roles  are  becoming  integrated  directly  with  like  people  from 
prime  contractors.  Test  pilots  from  Grumman  fly  flight  tests  in  Patuxent  River  side  by 
side  with  Navy  test  pilots  assigned  to  Pax  River  and  vice-versa.  Boeing  test  pilots  fly 
with  Navy  test  pilots  in  Pax  River,  trading  off  sorties  in  the  F/A-18E/F.  Navy  Joint 
Primary  Aircraft  Trainer  Systems  project  officers  (test  pilots)  fly  with  Air  Force  and 
Raytheon  test  pilots  at  Beech  Wichita. 

Is  it  perfect?  No.  Is  teaming  the  answer  to  every  programmatic  issue?  No.  While 
teaming  can  be  an  effective  means  of  getting  to  the  heart  of  a  problem  and  solving  it, 
teaming  can  create  some  problems  too.  Teams  must  be  careful  not  to  make  decisions  “in 
committee.”  These  decisions  can  often  be  a  compromise  that  is  one  of  expedience  rather 
than  consensus,  creating  less  than  optimal  designs  or  solutions  to  existing  problems. 
Team  training  can  be  expensive  and  time  consuming.  As  team  members  leave  and  are 
replaced  by  new  members,  this  training  must  usually  be  repeated  by  the  entire  team  to 
fully  incorporate  that  new  member.  There  are  still  many  people  who  resist  change  and 
want  complete  control  over  programs.  There  are  still  those  that  see  no  value  in  flight  test. 
There  are  still  hard  feelings  between  teams  as  a  result  of  problems  from  the  past.  There 
are  still  “old  timers”  in  program  offices  who  view  the  test  pilot  and  the  test  community  as 
“those  test  guys.”  There  are  very  functional  teams  and  nonfunctional  teams.  For  a  team 
to  work  properly,  all  members  have  to  “buy  in”  to  the  ‘no  vote’  concept  and  trust  the 
abilities  of  the  members.  All  members  of  the  team  must  understand  the  constraints  the 
team  must  operate  under,  especially  financial  constraints,  and  technological  limitations 
and  work  towards  those  ends.  Most  importantly,  all  members  of  the  team  must  have  the 
dedication  to  complete  the  program  or  kill  the  program  (whichever  is  required)  based  on 
the  needs  of  the  fleet  user. 

Flight  test  in  Patuxent  River  is  making  positive  and  dramatic  moves  to  incorporate 
the  team  concept  into  all  flight  test  programs.  “Up  front  and  early”  has  replaced  “kick  it 
over  the  fence”  as  the  basic  flight  test  philosophy.  In  a  world  where  tradeoffs  between 
cost,  schedule,  and  performance  drive  the  entire  acquisition  process,  the  team  concept  is 
gaining  strength.  The  product  continues  to  be  better  and  better  as  potential  problems  are 
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averted  on  many  programs  earlier  and  earlier.  Early  problem  identification  and 
resolution  are  saving  millions  of  dollars,  and  are  enabling  a  much  higher  mission 
performance  and  safer  aircraft  for  the  flight  test  program  and  the  end  fleet  user. 

There  are  still  many  challenges  ahead.  Teaming  is  an  evolutionary  concept  whose 
time  has  come,  but  the  evolution  is  not  complete.  A  few  more  years  are  required  to 
institutionalize  the  changes  developed  over  the  last  decade.  While  the  fleet  continues  to 
press  crew  concept,'  CRM/ACT,  and  teaming  throughout  the  various  communities, 
teaming  has  certainly  found  a  home  in  Navy  flight  test  and  indeed  the  entire  acquisition 
community.  With  continued  reductions  in  programmatic  cost  and  schedule,  and  increases 
in  system  safety  and  performance  that  teaming  brings  to  the  table,  it  is  definitely  here  to 
stay. 
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